A variety of video special effects can be enhanced or made possible by using a video camera that captures depth information in addition to color component information. The “ZCam”™ product able from 3DV Systems, Santa Clara, Calif., is a module that can be added on to a conventional studio video camera to provide depth information for objects in a scene captured by the video camera. The ZCam add-on relies on a sensing array that is separate from the color sensing circuit and thus entails a high cost. It could also be contemplated to integrate depth information pixels in arrays of red, green and blue pixels, but this also entails additional costs. That is, a camera which included an array or red, green, blue and depth pixels would have less density and hence less resolution and thereby higher cost for a given number of color pixels. In addition, the alignment problem that is generally encountered with R,G,B arrays is exacerbated because the interpolation routine for turning spatially separate R,G,B pixels into a single “RGB” pixel must also contend with an additional pixel for depth information.